1. Field of the Invention
The present invention relates to a machine for learning and recognizing input pattern data.
2.The Related Art of the Invention
A prior art example of a pattern recognition machine for recognizing input data by implementing major classification on the input pattern data to select a category group to which the input data belongs and then by implementing fine classification in the selected category group is described, for example, in "Large Scale Neural Network `CombNET-II`" Journal of the Institute of Electronics and Communication Engineers of Japan D-11, Vol. J75-D-11, No. 3 pp.545-553.
FIG. 1 illustrates a structure of the prior art pattern recognition machine, wherein a major classification section 101 coarsely classifies an input pattern signal into each category group. Fine classification sections 102 finely classify the input pattern signal into each category within the each category group. Some categories of one category group often overlap with some categories of another category group. A group selecting section 103 selects a plurality of category groups from the output values (hereinafter referred to as fidelity) of the major classification section 101. A fine classification section selecting section 104 selects the fine classification section 102 into which the input pattern signal is inputted based on the group selection information obtained from the group selecting section 103. A discriminating section 105 discriminates the input pattern signal from the fidelity of the category group selected in the group selecting section 103 and the output values of the fine classification section 102.
Input sections 106 in the major classification section 101 input the input pattern signal and multi-input/one-output(hereinafter referred to as multinput-output) signal processing sections 107 calculate the fidelity of each category group to the input pattern signal.
Input sections 108 in the fine classification section 102 input the input pattern signal output from the fine classification section selecting section 104. Multiinput-output signal processing sections 109 multiply the outputs of the under layer input sections 108 connected thereto or of the multiinput-output signal processing section 109 with weight factors which represent a degree of connection between them, respectively, and output the sum thereof after implementing threshold processing thereon. A degree of similarity to each category in the category group of the input pattern signal is found by connecting the plurality of multiinput-output signal processing sections in a network so as to have a layer structure, to have no connection mutually within each layer and to propagate signals only to the upper layers. A maximum value selecting section 110 selects the maximum value among outputs of the plurality of multiinput-output signal processing sections in the upper most layer.
Similarity calculating sections 111 in the discriminating section 105 calculate a similarity of each category from the fidelity of the category group selected by the group selecting section 103 and the output value of the fine classification section 102 corresponding to the category group. A category discriminating section 112 discriminates the input pattern signal by finding the maximum value of each category obtained from the similarity calculating sections 111.
The operation of the prior art pattern recognition machine constructed as described above will be described below. An input pattern signal X which consists of n feature data of an object to be recognized EQU X=(x.sub.1, x.sub.2 . . ., x.sub.n) (1)
is input to the input sections 106 of the major classification section 101 at first. The input sections 106 are prepared by n pieces equally to the number of feature data of the pattern data and each feature data x.sub.i is input respectively to the corresponding input section 106. Each multiinput-output signal processing section 107 in the major classification section 101 multiplies an input x.sub.j of the input section 106 connected thereto with a weight factor v.sub.ij (1.ltoreq.i.ltoreq.m.sub.r, m.sub.r is the number of category groups, 1.ltoreq.j.ltoreq.n) which represents a degree of their connection, calculates the sum of them, divides the sum by the product of norms .vertline.X.vertline. and .vertline.V.sub.i .vertline. of the input pattern signal X and a weight factor vector v.sub.i (Exp. 2) of each multiinput-output signal processing section 107 and outputs the result. EQU Vi=(v.sub.i1, v.sub.i2 . . ., v.sub.in) (2)
That is, the output value sim (X, V.sub.i) of the multiinput-output signal processing section 107 which has the weight factor vector V.sub.i may be expressed as follows: EQU sim(X, V.sub.i)=(X * V.sub.i)/(.vertline.X.vertline. .vertline.V.sub.i .vertline. (3)
where, X * V.sub.i =.SIGMA.j(x.sub.j , v.sub.ij)
.vertline.X.vertline.=(.SIGMA.x.sub.j.sup.2).sup.1/2
.vertline.V.sub.i .vertline.=(.SIGMA.v.sub.ij.sup.2).sup.1/2
Here, .SIGMA.j represents the sum of j.
By the way the weight factor vector V.sub.i is designed beforehand so that a predetermined multiinput-output signal processing section generates the maximum output to a similar input pattern signal.
According to the prior art examples, such weight factor vector V.sub.i is designed by the following method. In a first process, Vc whose sim (X, VV.sub.i) is largest (this case is called that X matches to Vc in optimum) is found to approach Vc to X every time when the input pattern signal X for designing weight factor vector is input. When input pattern signals which match to one weight factor vector in optimum reach more than a certain number, an area which the weight factor vector covers is divided into two. In a second process, V.sub.i which matches in optimum to all the input pattern signals for designing weight factor vector is found and is checked if it has changed from the previous one. If there has been a change, the V.sub.i is modified. At this time, the weight factor vector is divided similarly to the case in the first process. This process is repeated until the modification and division of the weight factor vector end.
The input pattern signal is coarsely classified into a plurality of category groups by thus designing the weight factor vector. The output value of each of the multiinput-output signal processing sections 107 is output to the group selecting section 103 as a fidelity of each category group to the input pattern signal X.
The group selecting section 103 selects an arbitrary number of category groups in an order from those having a large fidelity obtained in the major classification section 101 and outputs group selection information indicating which category groups have been selected and corresponding fidelities.
The fine classification section selecting section 104 selects the fine classification sections 102 to which the input pattern signal is input based on the group selection information obtained from the group selecting section 103 and outputs the input pattern signal to those fine classification sections 102.
In each of the fine classification sections 102, which correspond to the category groups selected in the group selecting section 103 (i.e., the fine classification sections to which the input pattern signal has been input from the fine classification section selecting section 104), the input pattern signal X is input to the input sections 108. The input sections 108 are prepared by n equally to the number of feature data of the pattern signal, and each feature data x.sub.i is input to corresponding input section 108, respectively. Each of the multiinput-output signal processing sections 109 in the fine classification section 102 multiplies the output of the underlayer input section 108 connected thereto or of the multiinput-output signal processing section 109 with the weight factor which represents a degree of their connection, respectively, transform the sum thereof by a threshold function and then outputs the resultant value to the upper layer. Here, the multiinput-output signal processing sections 109 in the most upper layer in each fine classification section 102 are set to be the same number with the number of categories of pattern data contained in each category group and each of the multiinput-output signal processing sections 109 in the upper most layer corresponds to each of those categories. The maximum value selecting section 110 selects the maximum value among the output values of each of the multiinput-output signal processing sections 109 in the upper most layer and outputs the category that corresponds to the multiinput-output signal processing section 109 and the maximum output value thereof.
By the way, the weight factor of each multiinput-output signal processing section 109 has been learned beforehand so that the multiinput-output signal processing section 109 in the upper most layer which corresponds to each category generates the maximum output for the input pattern signal having such category in the category group.
In concrete, such a weight factor learning-method is implemented by a learning algorithm called Back-Propagating Errors. The Back-Propagating Errors has been proposed, for example, by D. E. Rumelhart, G. E. Hinton and R. J. Williams in "Learning Representations by Back-Propagating Errors", Nature, vol. 323, pp. 533-536, Oct. 9, 1986.
An outline of the Back-Propagating Errors will be described below.
At first, a pattern signal X for learning weight factor is input to the input sections 108 of the fine classification section 102. As described before, each of the multiinput-output signal processing sections 109 multiplies the output of the underlayer input sections 108 respectively connected thereto or of the multiinput-output signal processing section 109 with the weight factor which represents a degree of their connection, respectively, transform the sum thereof by a threshold function and then outputs the resultant value to the upper layer. Here, an error E between output signals O.sub.k of all the multiinput-output signal processing sections 109 in the upper most layer and a desirable output signal t.sub.k (referred to as a teacher signal) may be found as follows: EQU E=0.5 .SIGMA..sub.p .SIGMA..sub.k (t.sub.k -o.sub.k).sup.2 ( 4)
Where, .SIGMA..sub.p is the sum of the number of patterns of the teacher signal. The purpose of the learning is to determine a value of the weight factor that minimizes the error E, and modification amount .DELTA.W.sub.ij of the weight factor among each multiinput-output signal processing section 109 is calculated based on the following expression: EQU .DELTA.W.sub.ij =-.epsilon..differential.E/.differential.W.sub.ij( 5)
Where, .epsilon. is a positive constant called a learning rate. The error E may be reduced by repeating the update of weight factor based on such Expression (5) every time when a pattern signal for learning is input. When the error E becomes sufficiently small, the learning is finished assuming that the output signal has sufficiently come close to the desired value.
Such weight factor learning method allows each multiinput-output signal processing section 109 in the upper most layer which corresponds to each category to generate the maximum output for the input pattern signal having such category in the category group. Accordingly, the category of the input pattern signal may be recognized in each category group or in each fine classification section by selecting one that generates the maximum output among the plurality of multiinput-output signal processing sections 109 in the upper most layer by the maximum value selecting section 110.
In the discriminating section 105, the similarity calculating sections 111 first calculate the similarity of each category obtained in the fine classification section 102 from the fidelity of the category group selected by the group selecting section 103 and the output value of the fine classification section 102 that corresponds to that category group, using Expression 6, and output those similarities to the category discriminating section 112. EQU (Similarity)=(Fidelity).sup.a (Output value).sup.b ( 6)
Where, a and b are real constants.
Finally, the category discriminating section 112 compares the similarities of each category obtained from the similarity calculating sections 111 and outputs the category that corresponds to the similarity which is largest among them as a discrimination result.
In the structure as described above, however, the learning of each fine classification section is carried out totally independently in the single fine classification section without considering a degree of belonging of the input pattern signal to the category group, i.e. a group belongingness, so that recognition accuracy of an input pattern situated at the boundary between category groups is degraded.
Furthermore, there has been a problem that when an unlearned pattern has been erroneously recognized and that pattern is to be learned again (i.e., supplemental learning), the supplemental learning cannot be implemented because the category of the pattern may not be belonging to the fine classification section in which the group belongingness has been determined to be maximum in the major classification section and hence that no determination can be made as to which fine classification section should be learned.